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The  purpose  of  this  study  was  to  determine  baseline  user  satisfaction  for 
2  computer  decision  support  systems  (DSSs)  with  demonstrated  improvement 
in  patient  outcome  used  in  a  burn  intensive  care  unit.  We  conducted  a 
survey  of  staff  members  of  a  16  bed  bum  intensive  care  unit  (n  =  82)  using 
a  written,  anonymous  questionnaire  to  determine  satisfaction  for  2  DSSs: 
a  commercial  glycemic  management  system  and  software  program  to  guide 
initial  bum  fluid  resuscitation.  Staff  members  are  not  yet  convinced  of  a 
positive  correlation  between  DSS  technology  and  patient  outcomes.  We 
suggest  user  satisfaction  may  be  generally  improved  for  DSS  with 
concentration  in  the  areas  of  interface,  information,  and  communication. 
Keywords:  Clinician  satisfaction,  Computer  decision  support  systems, 
Intensive  care  unit 
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The  introduction  of  computer  decision  support  systems 
(DSSs)  in  the  clinical  arena  offers  the  opportunity  for  stan¬ 
dardized  care  and  potential  for  improved  patient  outcomes. 14 
Computer  DSSs  have  been  shown  to  improve  individual 
clinicians’  performance  by  up  to  64%. 4  The  goal  of  DSS 
is  to  provide  clinicians  with  realtime  data  and  recommen¬ 
dations  that  guide  their  decision  making  process;  however, 
clinician  acceptance  is  a  vital  component  of  DSS  success. 

We  recently  adopted  2  computer  DSSs  in  our  burn 
unit,  one  to  guide  insulin  infusion  for  glycemic  control 
(EndoTool;  Hospira,  Inc,  Lake  Forest,  Illinois)  and  the 
other  to  direct  initial  fluid  administration  for  bum  injury 
(Burn  Resuscitation  DSS;  US  Army  Institute  of  Surgical 
Research,  San  Antonio,  Texas).  The  EndoTool  uses  the 
patient’s  previous  5  serum  glucose  levels  and  amount  of 


insulin  required  to  attain  those  levels  to  predict  the  amount 
of  insulin  required  to  achieve  a  target  glucose  range  de¬ 
fined  by  the  medical  director  (Figure  1).  In  a  similar  fash¬ 
ion,  the  Bum  Resuscitation  DSS  uses  the  patient’s  previous 
3  hourly  urinary  outputs  to  determine  the  amount  of  crys¬ 
talloid  infusion  required  to  achieve  a  target  urinary  out¬ 
put  in  the  subsequent  hour  (Figure  2).  These  systems  are 
available  at  the  computer  terminal  located  at  each  pa¬ 
tient’s  bedside  electronic  medical  record;  they  operate  from 
a  mainframe  server  located  elsewhere  in  the  hospital. 
Both  DSS  programs  are  considered  open-loop  systems  in 
that  the  user  can  accept  or  decline  the  generated  rec¬ 
ommendation.  Such  control  is  essential  as  the  clinician 
understands  the  complete  clinical  picture  for  a  patient, 
yet  limited  information  is  entered  into  these  DSSs. 
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Figure  1.  Sample  display  for  the  EndoTool  glycemic  computer  decision  support  system  (Hospira  Inc).  The  graph  displays  each  blood 
glucose  measurement  over  time.  The  highlighted  area  represents  the  target  range  programmed  for  the  patient  (80-110  mg/dL  glucose). 
Moving  the  cursor  over  the  measurement  will  display  the  corresponding  insulin  infusion  amount. 


Both  DSSs  have  demonstrated  improved  outcomes  in 
our  burn  intensive  care  unit  (ICU).  The  EndoTool  achieves 
our  target  glycemic  range  of  80  to  110  mg/dL  of  glucose 
more  often  than  the  traditional  paper-based  insulin  ti¬ 
tration  protocol  it  replaced,  while  achieving  significantly 
fewer  hypoglycemic  episodes.5  During  the  initial  48-hour 
burn  fluid  resuscitation,  we  have  shown  a  significant  re¬ 
duction  in  crystalloid  requirements  during  Burn  Resusci¬ 
tation  DSS  assisted  therapy  compared  with  the  previous 
practice  of  solely  provider-guided  resuscitation,  while  pro¬ 
viding  adequate  fluid  replacement.6 

Because  one  system  is  a  commercial  product  (EndoTool), 
and  the  other  is  a  software  program  developed  by  our  own 
computer  engineering  team  (Burn  Resuscitation  DSS),  we 
were  interested  if  the  clinical  users  perceived  a  difference  in 
satisfaction  between  these  2  systems.  The  purpose  of  this 
study  was  to  determine  baseline  user  satisfaction  for  the  2 
computer  DSSs  utilized  in  our  burn  ICU. 

B  METHODS 

This  study  was  approved  by  the  local  institutional  review 
board.  Recruitment  was  conducted  in  a  large  southwest¬ 
ern  level  I  trauma/  burn  center  in  a  16-bed  burn  ICU.  Cur¬ 
rently,  2  DSSs  are  utilized  in  our  unit:  a  leased  commercial 
product  used  to  guide  insulin  titration  (EndoTool;  Hospira, 


Inc),4  which  was  implemented  in  January  2009,  and  DSS 
software  developed  in  our  institution  to  assist  initial  burn 
fluid  resuscitation  (Burn  Resuscitation  DSS),5  which  was 
implemented  in  November  2007.  Training  on  EndoTool 
was  provided  by  Hospira  Inc,  consisting  of  40  hours  of 
both  didactic  and  hands-on  experiences.  In  contrast,  training 
for  Burn  Resuscitation  DSS  was  informal,  ranging  from  in¬ 
services  to  one-on-one  instruction  with  the  unit  clinical  nurse 
specialist  and  software  developers.  The  target  population 
for  training  on  both  systems  was  clinical  staff  who  utilized 
DDS  in  daily  practice  to  include  medical  doctors,  registered 
nurses  (RNs),  and  licensed  vocational  nurses  (LVNs). 

Data  collection  was  completed  by  distributing  a  written 
questionnaire  to  all  clinical  staff  (n  82).  The  participants 
were  encouraged  to  complete  the  survey  and  return  them 
anonymously  to  envelopes  centrally  located  in  the  burn 
unit.  Eleven  questions  utilizing  a  5-point  Likert  scale 
(where  1  “not  at  all,”  3  “neutral,”  and  5  “very 
much”)  were  asked  regarding  clinical  satisfaction  of  both 
decision  support  computer  systems. 

Descriptive  analysis  was  performed  for  participant’s 
duration  of  ICU  experience,  time  working  at  our  burn 
center,  and  burn  resuscitation  experience.  A  1-sample  Stu¬ 
dent  t  test  was  utilized  to  compare  the  differences  of  mean 
scores  to  that  of  neutral  (Likert  scale  3).  In  addition,  a 
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Figure  2.  Home  screen  for  the  Bum  Decision  Support  System  (US  Army  Institute  of  Surgical  Research).  The  displays  indicate  (from  top  left 
comer  in  clockwise  order)  (1 )  action  area  for  next  scheduled  intervention;  (2)  cumulative  infusion  volume  over  the  48-hour  resuscitation 
period,  where  colors  change  from  green  to  amber  to  red  to  indicate  dangerous  fluid  levels;  and  (3)  houriy  fluid  recommendation  (white 
dot),  actual  infusion  volume  (top  bar),  and  corresponding  urinary  output  (lower  bar)  color  coded  to  represent  in  range  (green),  below  range 
(yellow),  and  above  range  (red)  Colors  are  not  shown  in  this  depiction. 


paired  Student  t  test  was  used  to  compare  Burn  Resusci¬ 
tation  DSS  scores  with  EndoTool  DSS  scores.  Significance 
was  accepted  forP  <  .05.  Data  analysis  was  completed 
by  utilizing  PASW  statistical  software  (v.  18;  SPSS  Inc, 
Chicago,  Illinois). 

I  RESULTS 

Forty  surveys  were  returned  from  our  staff  (49%  response 
rate,  40/82),  consisting  of  5  physicians  (12.5%),  26  RNs 


(65.0%),  and  9  LVNs  (22.5%)  (Table  1).  Of  the  11  ques¬ 
tions  asked  about  the  EndoTool  system,  only  4  were  found 
to  be  significant  from  neutral  (Likert  scale  3)  by  the 
RN,  LVN,  and  total  combined  respondents:  (1 )  adequate 
training,  (2)  comfort  with  the  system,  (3)  ease  of  use,  and 
(4)  trust  in  recommendation  (P  <  .001,  respectively).  A 
difference  for  the  Burn  Resuscitation  DSS  was  noted  by 
the  physicians  in  (1)  adequate  training,  (2)  comfort  with 
the  system,  and  (3)  a  lack  of  interference  with  work 


Demographic  Profile  of  Study  Participants 

MD 

RN 

LVN 

Total 

n  =  5/5  (100%) 

n  =  26/54  (48%) 

n  =  9/23  (43%) 

n  =  40/82  (49%) 

Employed  in  bum  center,  y 

5.6  (5)  (1  14) 

3.3  (3)  (0  13) 

3.1  (2.8)  (1  8) 

3.6  (3.3)  (0  14) 

ICU  experience,  y 

7.6  (6)  (2  18) 

8.3  (5.6)  (1  19) 

5.9  (5.7)  (1  15) 

7.7  (5.6)  (1  19) 

Resuscitation  with  DSS,  n 

17.5  (5)  (10  20) 

4(3.7)(0  12) 

7.3  (6)  (1  20) 

6.2  (6)  (0  20) 

Resuscitation  without  DSS,  n 

80.5  (90)  (2  200) 

9.6  (20)  (0  100) 

25.6  (35)  (0  100) 

20.5  (40)  (0  200) 

Abbreviations:  DSS,  decision  support  system;  ICU,  intensive  care  unit;  LVN,  licensed  vocational  nurse;  MD,  medical  doctor;  RN,  registered  nurse. 
Values  are  as  mean  (SD)  (range). 
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( P  <  .05,  respectively).  No  difference  from  neutral  was 
noted  for  either  system  for  (1)  improved  care  or  outcome, 
(2)  increased  time  in  target  range,  (3)  improved  under¬ 
standing  of  concept,  (4)  helpfulness  for  training,  or  (5) 
overall  satisfaction  (Table  2). 

When  the  2  systems  were  compared,  the  only  differ¬ 
ences  noted  were  that  the  EndoTool  system  was  perceived 
to  be  superior  to  Burn  Resuscitation  DSS  for  combined 
user  responses  in  (1)  adequacy  of  training,  (2)  comfort  with 
the  system,  (3)  ease  of  use,  and  (4)  trust  in  the  recom¬ 
mendation  (P  <  .001,  respectively)  (Table  2). 

■  DISCUSSION 

This  study  of  clinician  satisfaction  with  computer  DSS  used 
in  our  burn  ICU  has  revealed  a  general  preference  for  the 
commercial  glycemic  management  system  (EndoTool)  over 
the  unit-developed  Burn  Resuscitation  DSS  for  initial  burn 
fluid  resuscitation.  However,  when  physician  and  nursing 
groups  are  considered  independently,  the  physicians  were 
more  satisfied  with  the  training,  comfort  in  use  of  the 
system,  and  decreased  interference  with  work  with  the 
Burn  Resuscitation  DSS  than  EndoTool.  Several  reasons 
may  account  for  this  distinction,  the  first  being  that 
implementation  of  these  systems  in  our  ICU  differed.  The 
commercial  EndoTool  system  was  supported  by  a  pro¬ 
fessional  team  of  nurse  educators  who  provided  a  week- 
long  training  and  “go-live”  period.  All  patients  on 
intravenous  insulin  infusion  were  eligible  for  DSS  glyce¬ 
mic  management,  and  several  patients  were  started  on  the 
system  during  the  training  period.  In  contrast,  the  Burn 
Resuscitation  DSS  developed  within  our  unit  was  intro¬ 
duced  incrementally  over  a  period  of  several  months  as 
severe  burn  patients  were  admitted.  Informal  one-on-one 
training  was  accomplished  with  the  nurses  and  physicians 
caring  for  the  individual  patient  during  resuscitation,  and 
unitwide  training  was  not  initially  conducted. 

Large  burn  injury  requiring  fluid  resuscitation  is  rel¬ 
atively  uncommon  (approximately  1  per  week),  with  a 
maximum  duration  of  only  48  hours.  Thus,  not  all  staff 
members  were  familiar  with  or  had  extensively  used  the 
Burn  Resuscitation  DSS.  In  addition,  during  the  early 
stages  of  development,  the  software  was  evolving,  likely 
introducing  ambiguity  among  the  staff.  Users  who  lacked 
expert  knowledge  of  the  underlying  principles  of  burn 
resuscitation  may  have  found  this  approach  confusing. 
The  physicians  reported  greater  satisfaction  with  Burn  Re¬ 
suscitation  DSS  training  and  comfort  with  the  system,  per¬ 
haps  because  of  the  emphasis  in  soliciting  continual  feedback 
regarding  system  performance  during  development  and 
the  relatively  fewer  number  of  staff  physicians  (n  5)  com¬ 
pared  with  nurses  (n  77)  in  the  burn  ICU.  The  nursing 
staff  reported  greater  satisfaction  with  EndoTool  likely 
because  of  the  daily  use  of  this  system  during  the  routine 
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administration  of  intravenous  insulin,  perhaps  contribut¬ 
ing  to  the  increased  satisfaction  in  ease  of  use  reported  for 
this  system.  The  physicians  do  not  directly  interact  with 
either  DSS  as  far  as  entering  hourly  inputs  into  the  sys¬ 
tems  or  adjusting  infusion  rates;  thus,  the  neutral  re¬ 
sponses  for  “ease  of  use”  were  unremarkable. 

A  reason  for  the  possible  difference  in  trust  in  system 
recommendations  is  the  comfort  level  of  the  providers 
with  the  underlying  processes  being  managed.  As  noted, 
glycemic  management  is  a  common  practice  for  all  ICU 
nurses,  and  insulin  titration  is  generally  performed  on  an 
hourly  basis  for  extended  periods.  Understanding  the  gly¬ 
cemic  response  of  patients  to  changes  made  by  the  nurse 
can  occur  quickly,  even  for  the  novice.  However,  fluid  re¬ 
suscitation  in  severe  burn  injury  is  complicated  by  multiple 
factors,  to  include  concomitant  injury,  inhalation  injury, 
organ  dysfunction,  age,  and  degree  of  burn  depth.7  Fur¬ 
thermore,  the  algorithm  guiding  the  complex  process  of 
fluid  infusion  is  based  on  the  single  variable  of  urinary 
output  when  in  fact  multiple  physiological  processes  are 
part  of  the  overall  approach  to  burn  resuscitation.  Most 
nurses  have  limited  experience  with  the  initial  24  hours 
of  burn  resuscitation  because  of  the  infrequency  of  ad¬ 
missions  compounded  by  a  large  nursing  staff.  This  may 
account  for  some  of  the  perceived  dissatisfaction  with 
the  Burn  Resuscitation  DSS  compared  with  the  EndoTool 
system  that  may  be  a  result  of  user  bias;  medical  doctors 
are  more  comfortable  with  directing  burn  fluid  resuscita¬ 
tion,  and  nurses  are  at  ease  with  insulin  titration. 

During  our  analysis  of  these  results,  our  team  realized 
that  although  we  have  successfully  demonstrated  that  both 
DSSs  have  improved  patient  outcomes  (improved  glycemic 
control  with  fewer  episodes  of  hypoglycemias  and  de¬ 
creased  fluid  requirements  for  burn  resuscitation  with  im¬ 
proved  outcomes,6  this  information  had  not  been  formally 
presented  to  the  staff  nurses.  Continued  feedback  of  the 
successes  of  new  technology  introduced  to  the  ICU  is  critical 
to  ensuring  staff  acceptance  and  trust  in  recommendations 
made  by  the  computer  systems.  The  neutral  responses  to 
the  question  regarding  whether  the  systems  resulted  in 
greater  time  in  target  range  or  improvement  in  patient  out¬ 
come  may  reflect  this  disconnect  with  the  finding  of  our 
outcome  research.  As  a  result  of  this  realization,  we  have 
made  a  concerted  effort  to  aggressively  promote  the  dem¬ 
onstrated  benefits  of  both  DSSs  in  the  care  of  our  burn 
patients  to  increase  our  clinician’s  satisfaction. 

Finally,  the  neutral  response  regarding  overall  user 
satisfaction  with  both  DSSs  provides  an  opportunity  to 
improve  our  communication  with  the  clinical  staff.  We 
have  developed  a  multifaceted  strategy  for  improving  ac¬ 
ceptance  of  new  technology  in  our  burn  ICU,  broadly 
targeting  the  areas  of  interface,  information,  and  commu¬ 
nication  (Table  3).  Our  clinical  nurse  specialists,  who  have 
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Strategies  to  Improve  User  Acceptance 
of  Computer  Decision  Support 
Systems  (DSSs) 

1 .  Interface 

•  Elicit  user  feedback  on  a  routine  basis  development,  deployment, 
and  ongoing  operations 

•  Programmers  would  appreciate  at  least  2  hours  per  week  from  routine 
user  to  assist  in  product  development 

•  Rewards  for  user  audible  tones,  levels  of  achievement,  points 
toward  a  goal  similar  to  video  games 

•  Keep  some  kind  of  patient  information  readily  available  to  avoid 
mistakes  in  patient  selection 

•  Develop  a  "training  module"  for  new  users  and  ongoing  education; 
demonstrate  system  processes  to  allow  practice  for  user  interface 

2.  Information 

•  Keep  it  simple:  give  a  brief  overview/outline  of  the  underlying  algorithm 

Example  for  glycemic  control 
o  1.  Enter  blood  glucose 

o  2.  System  projects  next  hour  insulin  infusion  rate 
o  3.  System  makes  recommendation  based  on  user  input 
o  4.  Wait  recommended  time  for  next  input 

•  Provide  explanations  for  DSS  recommendations  "why" 

•  Ensure  user  has  grasp  of  underlying  process  DSS  is  guiding 
(example:  principles  of  glycemic  control  or  burn  resuscitation) 

3.  Communication 

•  Inform  user  where  the  system  "is"  or  where  the  system  "is  going" 

•  Provide  mechanism  for  ongoing  user  feedback  to  programmers/ 
development  team  DSS  rs  an  iterative  development  process, 

a  "living"  application 

•  Provide  users  with  effect  of  DSS  on  patient  outcomes  improve  user 
buy  in  to  the  initial  inconvenience  to  workflow  with  improvement  in  care 


traditionally  been  the  leaders  in  introducing  and  mod¬ 
ifying  the  computer  systems,  are  the  primary  initiators 
of  coordination  among  the  providers  and  the  software 
developers.  In  fact,  we  have  noted  improvement  in  bidi¬ 
rectional  communication  among  the  nursing  staff  and 
physicians,  stimulating  discussions  regarding  best  prac¬ 
tices  and  ways  to  standardize  patient  care.  The  Burn  Re¬ 
suscitation  DSS  record  serves  as  a  useful  means  to  review 
difficult  resuscitations  during  interdisciplinary  rounds  and 
provides  teaching  opportunities  for  new  staff  members.  Fu¬ 
ture  satisfaction  surveys  are  planned  to  ensure  our  interven¬ 
tions  are  effective  in  providing  feedback  to  the  software 
developers  to  inform  and  improve  the  systems.  Such  sur¬ 
veys  will  provide  guidance  for  future  research  and  impli¬ 
cations  of  incorporation  of  computer  technology  in  clinical 
care  of  ICU  patients. 
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